International image or video coding standards like JPEG, MPEG-1/2/4 and H.261/H.263/H.264 use hybrid coding, wherein a picture is separated into pixel blocks on which predictive coding, transform coding and entropy coding is employed.
Normally, the transform coding is effective because the prediction error samples are correlated and in the transformed or frequency domain the signal energies become concentrated in partial areas of the coefficient blocks. Therefore any redundancy can be easily removed in the frequency domain. However, as disclosed in M. Narroschke, “Extending H.264/AVC by an adaptive coding of the prediction error”, Proceedings of Picture Coding Symposium, April 2006 (PCS2006), when the prediction quality is getting better and better, transform coding is no longer effective in many cases because the prediction error sample values are correlated only marginally and the signal energies will not concentrate in the frequency domain. M. Narroschke proposes a spatial domain or time domain video coding in which the prediction error samples (also called ‘residues’) are directly quantised and entropy-coded, without a prior transform into the frequency domain. He further proposed to use a rate-distortion optimisation (RDO) strategy for selecting adaptively whether to use spatial domain residue coding or transform coding.
FIG. 3 shows a corresponding sample block of ‘gradients’, the resulting block of (quantised) samples, and a scanning path. Although additional side information is required to indicate which coding type is used for a current block, the overall coding performance gain is significant. It is reported that the Y_PSNR gain can be improved by 0.4 dB compared with the H.264/AVC High profile, especially for CIF/QCIF format, and by 0.2 dB for SD/HD video sequences. It is also reported that upon combination ⅛ pel motion compensation, there will be another 0.5 dB gain for CIF/QCIF resolution. This technology was also proposed to the VCEG standard workgroup as M. Narroschke, H. G. Musmann, “Adaptive prediction error coding in spatial and frequency domain for H.264/AVC”, ITU-T, Question 6/SG16, document VCEG-AB06, Bangkok, Thailand, 16-20 Jan. 2006.
Encoding quantised samples in the time domain is also disclosed in H. Schiller, “Prediction signal controlled scans for improved motion compensated video coding”, ELECTRONICS LETTERS, 4 Mar. 1993, Vol. 29, No. 5.
In the above publications, the scan of the quantised samples in the spatial domain is carried out according to the magnitude of the gradient in the prediction image, i.e. in the reconstructed reference frame, at the same spatial position.
M. Narroschke, H. G. Musmann, “Adaptive prediction error coding in spatial and frequency domain with a fixed scan in the spatial domain”, ITU-T, Question 6/SG16, document VCEG-AD07, Hangzhou, China, October 2006, discloses a fixed scan in the spatial domain.